In the current development of range extension (RExt) for High Efficiency Video Coding (HEVC) standard, several tools have been adopted due to their improvements in coding efficiency for screen contents. In particular, the residue differential pulse-code modulation (RDPCM) and residue rotation (RR) tools are utilized for video compression. These tools may be applied to lossless transform-quantization bypass blocks or lossy transform-skip (TS) blocks. The “transform skip” coding mode is a coding mode where the transform process is skipped so that the residual is directly quantized and entropy coded. While transform is skipped for a block, other processing for a transform-coded block is still applied to the block. Accordingly, scaling is still applied to the TS block. The two-dimensional block residue needs to be converted into a one-dimensional signal. Accordingly, scanning (e.g., vertical or horizontal scan) is applied to the residue block. The scanning may also be applied before or after scaling, or after quantization. At the decoder side, the inverse TS (inv-TS) will be applied to a coded block. The inv-TS includes inverse scaling. It is noted that the coding unit (CU) is a basic unit that the coding process is applied. The residue is formed for each CU, which is further divided into blocks named transform units (TUs) for transform process used in conventional coding process (i.e., non TS). Furthermore, it is also possible to skip both the transform and the quantization processes to enable lossless coding of CUs. In the case of lossless coding, loop filtering is also turned off.
For Intra blocks, Intra prediction is performed using prediction based on reconstructed pixels from neighboring blocks. Intra prediction may select an Intra Mode from a set of Intra Modes, which include a vertical mode, horizontal mode and various angular prediction modes. For Intra blocks other than the blocks coded by Intra-block copy (IntraBC), the direction of RDPCM processing depends on the Intra Mode of the underlying block. RDPCM is only applied to Intra coded blocks using the horizon and vertical modes along the respective horizontal and vertical directions. For Inter and IntraBC blocks, the encoder decides whether to use RDPCM by signaling a flag. The encoder also decides the RDPCM direction by signaling another flag. While RDPCM for both Intra residue (other than IntraBC) and Inter residue uses horizontal DPCM or vertical DPCM, there are some differences between Intra RDPCM processing and Inter RDPCM processing. The selection of prediction direction (i.e., horizontal or vertical) for Intra RDPCM processing is determined implicitly based on the Intra Mode for the block. On the other hand, whether to use RDPCM and the prediction direction are determined at the encoder side and the decision is signaled explicitly. In this disclosure, RDPCM processing refers to the DPCM (Differential Pulse-Code Modulation) procedure applied to a residue signal or processed residue signal (e.g., a residue signal processed by TS). RDPCM coding refers to coding method that includes RDPCM processing.
FIG. 1 illustrates an exemplary of lossless vertical RDPCM processing, where the residue samples of an Intra coded block are subtracted with a vertical neighboring sample except for the samples in the top row to further reduce the redundancy. For a block with N columns and M rows of samples, let ri,j, 0≤i≤M−1,0≤j≤N−1 be the prediction residue after Intra prediction. Assume that the transform and quantization are skipped. Lossless vertical RDPCM processing is applied to the residual samples as follows:
                                          r            ~                                i            ,            j                          =                  {                                                                                                                r                                              i                        ,                        j                                                              ,                                                                                                              i                      =                      0                                        ,                                          0                      ≤                      j                      ≤                                              (                                                  N                          -                          1                                                )                                                                                                                                                                                                            r                                                  i                          ,                          j                                                                    -                                              r                                                                              (                                                          i                              -                              1                                                        )                                                    ,                          j                                                                                      ,                                                                                                              1                      ≤                      i                      ≤                                              (                                                  M                          -                          1                                                )                                                              ,                                          0                      ≤                      j                      ≤                                              (                                                  N                          -                          1                                                )                                                                                                                  .                                              (        1        )            
For RDPCM coding, the DPCM processed residue (i.e., {tilde over (r)}i,j) is further coded instead of the residue signal (i.e., ri,j) itself. For lossless horizontal RDPCM processing, the subtractions are performed between a current sample and a left neighboring sample except for samples in the first column. As for the lossy case, the residue differences are subjected to quantization. Equation (2) illustrates an example of lossy vertical RDPCM processing being applied to the residual samples:
                                          r            ~                                i            ,            j                          =                  {                                                                                                                r                                              i                        ,                        j                                                              ,                                                                                                              i                      =                      0                                        ,                                          0                      ≤                      j                      ≤                                              (                                                  N                          -                          1                                                )                                                                                                                                                                                                            r                                                  i                          ,                          j                                                                    -                                              Q                        ⁡                                                  (                                                      r                                                                                          (                                                                  i                                  -                                  1                                                                )                                                            ,                              j                                                                                )                                                                                      ,                                                                                                              1                      ≤                      i                      ≤                                              (                                                  M                          -                          1                                                )                                                              ,                                          0                      ≤                      j                      ≤                                              (                                                  N                          -                          1                                                )                                                                                                                  ,                                              (        2        )            where, Q(r(i-1),j) denotes a quantized version of residue sample, r(i-1),j. For Inter RDPCM processing, residual DPCM is applied to Inter coded blocks. Equation (1) is also applicable to Inter RDPCM processing, where the residue signal corresponds to the Inter prediction residue instead of the Intra prediction residue for the case of Intra RDPCM. For lossless horizontal RDPCM processing, the subtractions are performed between a current sample and a left neighboring sample except for samples in the first column. As for the lossy case, the residue differences are subjected to quantization.
As for residue rotation (RR), the residue block is rotated 180-degree so that it will be likely to have the likely larger DPCM prediction residue in the bottom-right corner rotated to the top-left corner. The rotated DPCM prediction residue will better fit for the entropy coder that was originally designed for transform coefficients. FIG. 2 illustrates an example of the RR processing, where the DPCM prediction residue has some non-zero (NZ) samples in the bottom-right corner. The NZ samples are rotated to the top-left corner by the RR processing.
According to the existing RExt practice, the encoding and decoding processes for the residue vary depending on Inter/Intra and lossy/lossless selections when both RDPCM and RR processing are invoked. Following is the highlight of the corresponding encoding and decoding processes for the residue signal. The notation “inv-TS” denotes the inverse transform-skip, and “inv-RDPCM” denotes the inverse RDPCM.
Note that RDPCM is invoked only under transform-quantization (transquant) bypass, or under transform-skip. Also note that Residue Rotation (RR) itself has a separate control flag, which can be enabled independent from RDPCM. In other words, when RDPCM is used, RR can be on or off; and when RR is used, RDPCM can be on or off.
Encoding Process
Intra (excluding Intra-block-copy) encoding:                If CU is coded in transquant bypass: RDPCM→RR        If TU is coded in transform-skip: RDPCM→TS→RR→quantization        
Inter and Intra-block-copy encoding:                If CU is coded in transquant bypass: RDPCM→RR        If TU is coded in transform-skip: TS→RR→RDPCM→quantization        
Decoding Process
Intra (excluding Intra-block-copy) decoding:                If CU is coded in transquant bypass: RR→inv-RDPCM        If TU is coded in transform-skip: De-quantization→RR→inv-TS→inv-RDPCM        
Inter and Intra-block-copy decoding:                If CU is coded in transquant bypass: RR→inv-RDPCM        If TU is coded in transform-skip: De-quantization→inv-RDPCM→RR→inv-TS        
In the above encoding and decoding process, “transquant bypass” refers to a specific lossless coding mode, where a lossless coding means bypassing both the DCT transforms and bypassing quantization is applied. The two processes as highlighted in bold, such as TS→RR and RR→inv-TS, indicates that they are bundled together. For software based implementation, RR can be performed within the TS and inv-TS functions. The decoding process inconsistency increases implementation difficulty. Therefore, it is desirable to harmonize and unify the decoding process for the combinations of RDPCM and RR.
In the existing RExt of HEVC, the Inter-RDPCM is applied only to Inter blocks. The IntraBC residue is formed between a current block and a reference block in the same picture is regarded as an Intra block. Therefore, Inter-RDPCM is not applied to blocks processed by IntraBC prediction mode. On the other hand, the blocks coded in IntraBC prediction mode doesn't use horizontal or vertical Intra Mode, therefore these blocks are not processed by Intra RDPCM either. It is desirable to develop coding scheme to improve performance for blocks processed by IntraBC prediction mode.